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Abstract 


To study beliefs about ability and STEM major choice, I conduct a field experiment where I 
provide students with information that they are above average in their top fields of study. I find 
that STEM students are more likely to switch out of their major and that non-STEM students 
fail to switch into STEM at the same rates as other fields. I also find that learning you are 
above average in your top field of study increases STEM major choice by almost a third, as 
STEM students appear more like to persist and non-STEM students increase their switching 
into STEM fields. 
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1 Introduction 


Despite a growing demand for STEM-capable workers in the U.S. workforce, college students in- 
tending to major in science, technology, engineering and mathematics (STEM) fields fail to do 
so at extremely high rates (Olson and Riordan, 2012; Xue and Larson, 2015; Chen and Soldner, 
2014). STEM students also are more likely to switch majors or drop out entirely when compared 
to students of other fields (Stinebrickner and Stinebrickner, 2013). This phenomenon is one version 
of what is often referred to the STEM “leaky pipeline”, in which students initially interested in 
completing a STEM degree either switch majors or drop out altogether. 

So far, the research on the causes of this leaky pipeline out of STEM is decidedly thin. 
Economists studying major choice more broadly have predominantly focused on students expecta- 
tions of post-graduation returns to majors, with an emphasis on pecuniary returns (Zafar, 2011; 
Wiswall and Zafar, 2015a, 2018, 2015b; Conlon, 2020). Relatively little attention, however, has been 
paid to students’ beliefs about their ability to do well in different majors. Beliefs about ability are 
likely to be relevant in studying the leaky pipeline as STEM fields are often considered some of the 
most academically demanding majors. Also, while incoming college students may have a sense of 
the overall ability distribution within an institution, they importantly have imperfect information 
about how they compare academically to successful STEM graduates!. 

In an important study of this topic, Stinebrickner and Stinebrickner (2013) conclude that as- 
piring STEM students fail to graduate with a degree in the those fields ultimately because they 
enter college with incorrect beliefs about their ability. The authors argue that, as students progress 
through the science curriculum, they receive negative shocks to their beliefs via grades in science 
courses, which tend to be lower compared to non-science fields, ultimately leading them to switch 
majors. For educators and policy-makers hoping to increase interest and persistence in STEM 
tracks, a natural question that arises in light of this is; if students were to receive a positive shock 
to their beliefs about their ability to complete a STEM degree, would we see a corresponding 


increase in STEM persistence? 


‘Using data provided to me from the UC Davis registrar on the average HS GPA, SAT and ACT scores of recent 
graduates in different majors, I see that graduates of STEM fields have average scores that are higher than either 
biology, social science or humanities graduates across all the measures just mentioned. 


To study this question, I conduct a field experiment in which I first elicit college students’ top 
choices of major and then randomly provide them with information they are above average in those 
fields. I then measure how this information influences their decision to major in STEM. In my 
descriptive analyses, I show that students are indeed more likely to leave their preferred field of 
study if that field is a STEM field compared to students whose top field is non-STEM. Specifically, 
STEM students are four times as likely to leave their top choice of major compared to non-STEM 
students. I also show that this is not due to STEM students placing less likelihood on graduating 
in that field compared to non-STEM students. STEM students believe they are just as likely to 
graduate in their top field as non-STEM students. When studying beliefs about ability, I also find 
that over one in four STEM students who are actually above average in their top field believe they 
are “below average” in that field. Focusing on those students who switch out of STEM, they are 
more likely to be female, believe that their peers are of higher ability relative to their own ability 
as well as overestimate the actual ability of their peers. 

In addition, my data allow me to identify a leaky pipeline into STEM fields, as students are 
less likely to switch into STEM majors if they are originally interested in a non-STEM field. This 
second “pipeline” into STEM is rarely discussed, if even mentioned, in the STEM major choice 
literature, although my results show that getting students to switch into STEM fields is likely as 
important to increasing STEM major choice overall as increasing persistence in those fields. 

In my experimental results, I find that the information I provide to students increases persistence 
in STEM by 40 percent among marginal out STEM students, those whose top choice of major is 
a STEM field, but who are also considering majoring in fields other than STEM. Focusing on 
marginal into STEM students, those who are considering majoring in STEM but do not select 
it as their top choice, I find my intervention also increased switching into STEM by a significant 
amount. Looking across the entire sample, I find that my treatment increased STEM major choice 
overall by five percentage points, representing close to 30 percent of the control mean. I also find 
that treatment effects on STEM major choice are strongest for first-generation students. 

Using unique data on beliefs that I elicit as part of my intervention, I find that the information 


I provide does not provide any differential impact for students based on the beliefs they held about 


mean ability ex-ante. This implies that, similar to effects of learning about returns to majors 
(Conlon, 2020), learning you are above average compared to recent STEM graduates likely induces 
changes in STEM major choice through reducing uncertainty rather than shifting mean beliefs about 
ability. Taken together, these results show that beliefs about ability are an important feature of 
STEM major choice. Learning you are above average in your top choice of major can help alleviate 
uncertainty about those beliefs and meaningfully inform subsequent decisions about which fields to 
major in and whether to continue on and ultimately graduate in that field. 

This paper contributes to several literatures. The first is the literature studying the determi- 
nants of college major choice. Most papers in this literature use structural modeling techniques to 
untangle the magnitudes of different determinants of students’ college major decisions, including 
expected earnings, productivity and academic ability (Arcidiacano et al., 2012; Arcidiacano, 2004; 
Gong et al., 2019; Zafar, 2011). Another branch in this literature studies the major choice decision 
by providing information to students about characteristics of majors and measuring changes in 
students’ major choice behavior (Conlon, 2020; Li, 2018; Carrell et al., 2020; Owen, 2020). Other 
work examines the effect of role models in major choice (Carrell et al., 2010; Porter and Serra, 
2019) 

Despite the various focuses, approaches and methods used, one commonality among these papers 
is that each describe perceptions of ability as an important feature of the major choice decision. 
Surprisingly, however, there has been little work studying this mechanism directly. In one of the few 
papers addressing this, Li (2018) finds that female students in an introductory economics course 
respond positively to learning about their ability. In this paper, the author finds a 50 percent 
increase in economics major choice as a result of their information and nudge treatment. In a 
similar study using information derived from introductory STEM courses, Owen (2020) finds that 
low-performing male students are likely to leave STEM as a result of learning how they compare 
to other students in their class, effectively reducing the gender persistence gap in STEM. I add to 
this line of literature by estimating causal effects on STEM major choice that arise from shocks to 
beliefs about ability across a large set of majors. This allows me to study both rates of persistence 


in as well as switching into STEM fields that may derive from beliefs about ability in non-STEM 


fields. 

Secondly, this paper contributes to the study of beliefs in education (Bobba and Frasinco, 
2019a,b; Zafar, 2011; Bleemer and Zafar, 2018; Dizon-Ross, 2019; Kaufman, 2014; Attanasio and 
Kaufman, 2014; Rury and Carrell, 2020; Ersoy, 2019; Conlon, 2020). Papers in this literature 
highlight the important role of beliefs in student decision-making across a range of outcomes, 
including study effort, college-going and major choice. As such, in order to fully understand the 
casual factors behind academic decision making, researchers must collect and analyze students’ 
expectations as observed decisions may be compatible with several different sets of preferences 
(Manski, 2004). In light of this, I collect and study several beliefs about students’ major choice. 
I use these beliefs to identify important features of students’ major choice decisions that would 
not be possible using observational data alone. Specifically, by collecting subjective data about 
different majors, I can better characterize the leaky pipelines out of as well as into STEM fields 
and link these phenomena to beliefs about ability. 

Thirdly, this paper contributes to the literature on performance feedback in education. Papers 
in this literature study the effect of learning about your performance on an academic assessment 
(eg. grade) relative to others on subsequent academic outcomes (Azmat and Iriberri, 2015; Azmat 
et al., 2019; Azmat and Iriberri, 2010; Bandiera et al., 2015; Murphy and Weinhardt, 2020; Goulas 
and Megalokonomou, 2018; Brade et al., 2018; Gonzalez, 2017). These studies find that providing 
feedback to students changes students’ perceptions of their ability or self-confidence (Murphy and 
Weinhardt, 2020). The information I provide students is a unique form of performance feedback 
that allows students the rare opportunity to compare themselves to other students within a field 
of study while in college. This information is highly salient to students, as the measures of ability 
I provide in my intervention are derived from statistics most students spend a great deal of time 
considering when applying to college; namely high school (HS) GPA, SAT and ACT scores. This 
paper is the first to assess how receiving performance feedback about academic performance derived 
completely before entering college influences college major choice. As such, my results speak directly 
to objectives of university administrators and policy makers hoping to bolster the academic success 


of those students about to enter college. It is also the first paper to provide feedback to college 


students across a set of majors. 

My results are also important for the study of labor supply in STEM occupations in the U.S. 
Along with a leaky pipeline found in STEM fields in higher education, there has also been a 
slowdown in the domestic supply of STEM-capable workers in the U.S. labor force (Carnevale 
et al., 2011). This slowdown has led policymakers to call for a drastic increase in the number of 
STEM-capable workers (Olson and Riordan, 2012). This paper studies a plausible explanation for 
this slowdown; that STEM majors leave STEM (and non-STEM students fail to switch into STEM) 
because of misperceptions of their ability to succeed in those majors. As I describe later, I find 
evidence that by providing students with information that they are above average in their top field 
of study increases STEM major choice, highlighting the potential for larger policy interventions 
aimed at addressing this slowdown in STEM labor supply. 

Lastly, I contribute to the literature studying the use of nudges in education settings. While 
nudges are often deployed in these settings, existing evidence is mixed on their efficacy in changing 
behavior (Oreopoulos et al., 2020; Oreopoulos and Ptronijevic, 2019; Damgaard and Nielsen, 2018). 
I show that light touch interventions can indeed induce meaningful behavioral change in educational 
decision making. Later on in the discussion, I consider what attributes of my intervention may be 
influential in changing behavior. 

The rest of the paper is structured as follows; section 2 describes the experimental design and 
setting, including a discussion of the information intervention and how it was constructed; section 3 
describes the data, presents the descriptive results and discusses the leaky pipeline out of and into 
STEM; section 4 presents the experimental results; section 5 provides a discussion of my findings 


and concludes. 


2 Experimental Details 


The experiment was conducted during the fall 2018 and winter 2019 academic quarters at University 
California at Davis (UC Davis). UC Davis is a selective public research university in Northern 
California with an undergraduate enrollment of over 30,000 students. The 75th percentile of HS 
GPA for the 2019 freshman class was 3.86, while the 25th percentile was 4.18. To study STEM 


major choice, the sampling frame was chosen to consist of large, introductory STEM courses. Course 
instructors and department administrators were contacted for participation during the preceding 
academic terms, with over a dozen instructors agreeing to advertise the survey. 

Several pieces of the experiment were arranged prior to students taking the survey. Firstly, 
to create my information treatment, the university’s registrar provided me with average scores 
of recent graduates within the a set of 16 major groups. These averages were calculated for five 
different measures of ability; HS GPA, SAT combined score, SAT math score, SAT reading and 
writing score and ACT score. These average scores were derived using scores from the five most 
recent cohorts of UC Davis graduates and were calculated within each major group. This implies 
that I was given 16 x 5 different measures of ability that were calculated using the universe of 
recent UC Davis graduates. These averages were incorporated into the survey and used as part of 
the survey’s internal display logic. Secondly, for all students included in the experiment’s sampling 
frame, scores from all five measures were also uploaded to be used within the survey”. Students 
from participating classes were incentivized to take the survey by being entered into a raffle to win 
one of several amazon gift cards. 

Upon taking the survey, participants were asked several questions about their educational pref- 
erences and beliefs. First, students were asked to identify their top and second choices of majors. 
Students were given 16 options; economics, biological sciences, physics, chemistry, communications, 
psychology, engineering (any type), mathematics, statistics, foreign language, computer science, En- 
glish, history, philosophy, political science and sociology. As most of the literature on major choice 
has focused on the pecuniary returns to majors, students were then asked about their expected 
earnings both five and 20 years after graduating in both their top and second choices of major. 

To capture students’ subjective expectations about graduating with different majors, student 
were also asked “what is the probability you will believe you graduate in your top choice of major, 
second choice of major, or some other major?”, with the sum of the probabilities constrained by 
the survey to sum to one. To study their beliefs about their relative ability in different fields, 


students were then asked a series of questions concerning their top choices of major. This series of 


?This was necessary due to a research design constraint from my campus partners that restricted me to only give 
information to students whose own score was above the average score. I discuss this further later on in the paper. 


questions was designed to avoid any direct elicitation of beliefs about ability, as these responses may 
have been more prone to experimenter bias or motivated reasoning. In my elicitation procedure, 
students were first asked to select which academic measure best represents ability in their top and 
second choices of major. Students were given the same options for those I was provided average 
scores of recent graduates; HS GPA, SAT, SAT math, SAT reading and writing and ACT score. 
After students selected these measures for both their top and second choice of major, they were 
then asked “What do you believe is the average score of recent UC Davis graduates is in your top 
choice of major?”, where “score” was replaced with the academic measure they selected and “top 
choice of major” with their selection for most preferred major. 

After students provided their beliefs of the average score of recent graduates in their top choices 
of major, the survey determined who is eligible to be randomized into either treatment or control 
groups. To do this, the survey compared the student’s own score in the measure they selected for 
their top choice of major with the average score of recent graduates in that measure and major. If 
this criterion was satisfied, students were then randomized into either treatment or control groups. 
For students randomized into the treatment group, a message appeared that read “The average 
score of recent graduates in your top choice of major is (actual average major-measure specific 
score)”, where again “score” is the measure students selected as most representative of ability and 
“top choice of major” is students selection for top choice of major. This message was then followed 
by a small nudge that stated “Our records show that your score is above the average score of 
graduates in your top choice of major. We hope that this information helps you in your college 
major decision.” The survey then followed the same procedure for the student’s second choice of 
major, after which the survey concludes. All survey items as well as the information I provided to 
participants can be found in the appendix. 

Due to design constraints imposed by my institutional collaborators, I was not permitted to 
provide information to students whose own score was below the average score of recent graduates. 
I discuss this restriction towards the end of the paper when I consider the policy implications of 
my findings. Lastly, students were also asked to sign a Family Educational Rights and Privacy Act 


(FERPA) release so that I could access their academic records, including their major choice history 


history, as well as their demographic and background characteristics. 


3 Data and Descriptive Results 


3.1 Data 


The data for this project consist of 728 completed baseline surveys, as well as demographic and aca- 
demic background information, and major choice history up to three years after the experiment for 
all students who signed a FERPA release. Due to the research design constraint described above, 
483 of these students were eligible for treatment in their top choice of survey, as they had scores in 
majors/measures they selected that were in fact above the average score of recent graduates in that 
major/measure °. Slightly fewer, 456, were eligible to receive information in their second choice of 


major“. 


3.2. Experimental Balance 


To ensure that my randomization procedure worked as designed, I conduct several tests. Through 
my collaboration with the university, I received a rich set of variables on student demographics 
and pre-collegiate academic performance from the university’s registrar, including race/ethnicity, 
gender, low-income and first-generation status as well as HS GPA and standardized test scores. 
I also observe each student’s major at time of admission to the university. Table 1 shows results 
from regressions that study the relationship between treatment status and survey responses and 
demographic variables for both top and second choice of major. We see that none of the coefficients 
from either model are statistically distinguishable from zero. I view this as strong evidence that 


students were in fact randomly assigned between treatment and control groups for treatment in 


3One clear question here is why is the number of students who are “above average” more than half of the sample. 
There are two answers, one slightly less obvious than the first. Firstly, this may represent sample selection, with those 
who are above average more likely to take the survey than those who are not. Another explanation come from the 
fact that standardized test and HS GPA scores at UC Davis have increased over the years. This would mechanically 
push up the number of students who are above average when compared to average scores of previous cohorts. 

“This was due to students not being above average in their measure/major for their second choice of major or not 
having a valid choice for second choice of major 


both a student’s top and second choice of major. 


3.3. Sample Descriptives 


Table 2 provides demographic information for students in my analysis sample’. The sample is 
almost exactly two thirds female, slightly higher than the proportion of undergrads that are female 
at UC Davis (61%). Almost half of the sample is Asian or Asian-American, a 30%, a sixth Hispanic 
and about 2% African American. While there are small differences, these numbers mirror the 
student population at UC Davis quite well®. Low-income and first-generation students, however, 
appear to be slightly under-represented in my sample when compared to the UC Davis population. 
They represent 21% and 32% of my sample respectively, while they constitute 37% and 42% of 
the student body. As my sampling frame consisted of large introductory STEM classes, 63% of 
students in my sample are freshman. 

Table 3 presents results on top and second choice of major for the full set of survey respondents. 
For these results, I aggregate majors into four groups; social sciences (economics, sociology, po- 
litical science, psychology); humanities (English, foreign language, history, philosophy); Biological 
Sciences; and STEM (physics, chemistry, mathematics, statistics, computer science and engineering 
(any type)). We see that biology is the most popular top choice of major (47.6% of students), fol- 
lowed by social sciences (24.6%), STEM (22.6%), and humanities (5.2%). The relative popularity 
of biology is striking, but somewhat consistent with major choices at UC Davis, with it’s strong 
agricultural focus and popular biochemistry fields’. 

Table 2 also presents descriptive statistics broken down by whether students select a STEM 
field as their top choice of major®. Here we see that STEM students are almost 9 percentage 


points more likely to select the math portion of the SAT as most representative of ability in those 


°This includes all students who were eligible for treatment in their top choice of major 

°The Asian/Pacific Islander population at UC Davis represents about 32% of the student population. Another 
17% of students are international, the vast majority of which are from Asian countries, predominantly China. The 
sum of these two terms is 49%, almost identical to the number of Asian students in my sample. Unfortunately, I do 
not have international status in my data. 

"The sampling frame for this study was introductory STEM courses, which ended up including two large intro- 
ductory biology courses. This fact may be skewing the distribution towards biology students 

8 As I discuss below, I separate biology from STEM in all of my descriptive analyses. 
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fields. These beliefs are consistent with stereotypes both of STEM students being math-focused 
and STEM curriculum being relatively math-heavy. We also see that STEM students expect to 
earn significantly more than non-STEM students, both five and 20 years after graduation. Looking 
at gender, I find that STEM students are 30 percentage points less likely to be female. Focusing 
on beliefs about ability, figure 2 plots the distribution of the difference between students belief of 
the average score of recent graduates in the top major/measure and their own score (own score - 
belief of average). Students who have a negative value for this variable believe the average score is 
above their own, effectively rating themselves as “below average” in this major. Figure 2 also plots 
this distribution for students whose own score is actually above the average score (which makes 
them eligible to receive information). Two facts emerge from inspecting these plots. First, about 
half of students who select STEM as their top field believe they are “below average”. Second, we 
see here that about 27% of STEM students who are actually above average believe they are below 
average (these students represent my primary analysis sample). All of these results are robust to 


controlling for majors within these groupings. 


3.4 The Leaky Pipelines Out of and Into STEM 


To explore the dynamics of major persistence and major switching over time, I perform several 
analyses. When doing so, I exclude both those who where ineligible for treatment (those who 
had scores below the average in their selected major/measure) as well as students who received 
information in their top choice of major from these analyses. I first compare the distribution of 
top choice of major to the distribution of major choices I see three years later®. These results 
on persistence can be found in Table 4. Due to the large number of students intending to study 
biology, I have separated it from other STEM fields and presented the three major groups along 
with biology. 

Looking at table 4, we see relative increases in the number of students majoring in biology, social 


sciences and humanities three years after the experiment compared to the number of students who 


°For students who are no longer enrolled at this point, I use their most recent major. As the control group were all 
students who were above average students in their top field of study, all departures from the university are through 
graduation and not dropping out. 
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selected those fields as their top choice of major at the time of the survey. While biology and the 
humanities stay about the same over this period (increases of 4.5% and 0% respectively), social 
sciences saw a 13.5% increase in the number of students over this time. In sharp contrast, however, 
STEM fields saw a decrease of 33.3% during this time. 

Perhaps a more direct way to assess whether students of different majors are more or less likely 
to persist in those fields is to see which of these groups of majors has the highest persistence 
rate, conditional on selecting it a major in that group as your top choice . To do this, I create a 
variable that equals one if the student is majoring in their top choice of major two years after the 
experiment. We see in table 5 that 92% of biology students and 88% of social science students are 
majoring in their top choice of major two years later. Looking at STEM students, however, only 
61% are still majoring in their top field !°. 

Given that STEM fields are highly mathematical and are known for “weeding out” students 
with insufficient aptitude in mathematics, these transitions out of STEM may reflect differential 
mathematical preparation. Instead of observing a leaky pipeline that reflects incorrect beliefs about 
ability, we may simply be observing the least mathematically prepared students switching to other 
majors. To study this, I compare the SAT math scores of those STEM students who switched out 
of their top field to those who persisted as measured three years after the intervention. Table 9 
presents descriptive comparisons between these “leakers” and “persisters”, namely those students 
who top choice of major was a STEM field, were randomized into control, and who ultimately did 
not (leakers) or did (persisters) major in a STEM field. While these samples are small (n = 31, n 
= 21 respectively), they provide a unique glimpse into the demographic makeup of students who 
switch out of STEM. Here we see that these students have an average SAT math score that is 14 
points (800 point scale) higher than those students who continue on in their top field. While this 
result is not statistically significant, it raises doubts about the role of mathematical aptitude in 
explaining exit out of STEM in this setting. 

Table 9 also shows that STEM students who switch majors are more likely to be female (sig- 


nificant at 10% level), first-generation students and Hispanic, although there is no statistically 


Only 57% of students who chose a field in the humanities are still majoring in that field two years later, although 
there are only 14 of these students so it is hard to draw sharp conclusions about these fields. 
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detectable difference for these last two characteristics. Focusing again on beliefs about ability, 
students who switch out of STEM are more likely to rate themselves as being relatively less aca- 
demically capable in those STEM fields (own score - belief of average) and are also appear more 
likely to overestimate the actual average score of recent graduates in STEM fields (actual average 
- belief of average). These differences, however, are not quite statistically significant. 

These dynamics out of STEM may not truly reflect a leaky pipeline if student expectations 
are in line with the distribution of major choices three years after the experiment. For example, 
students who begin with a STEM field as their top major may expect to switch out of that field 
more so than students whose top choice of major is another field. This might be true if students 
place a higher option value on starting college in a STEM field. As we saw earlier, students whose 
top choice of major was a STEM field expected to earn significantly more in those fields conditional 
on graduating in those majors. This would imply that students experiment with STEM, but place a 
lower probability on the likelihood that they ultimately persist in that field. If students were indeed 
as earnest in their top choice of major in STEM as in other fields, the differences in likelihoods of 
graduating in your top majors would not be very different between STEM and non-STEM fields. 

As part of the survey, students were also asked to provide the likelihood they believed they 
would graduate in their top choice, second choice or some other major. Table 6 in the appendix 
shows that vast majority (90.2 percent) of probability is placed on either the top or second choice of 
major, demonstrating that students are mostly considering majoring in either their top or second 
choice of major, rather than a larger set of majors. I incorporate these subjective probabilities 
into my analysis of the leaky pipeline to assess the following hypothesis; do students who intend 
to major in STEM place less probability on doing so than students of non-STEM fields? To do so, 
I condition on top choice of major group and compare the average subjective probability students 
report that they will graduate in that field to the average empirical probability they in fact do. 
Again here I restrict the sample to the control group for top choice of major. Table 6 presents 
the results. Here we again see strong evidence of a leaky pipeline out of STEM fields. STEM 
students appear to report subjective probabilities of graduating in their top field that are only 


slightly different than students of other fields. Importantly, however, STEM students are much less 
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likely to persist in that field two years after the survey than students of other fields. 

All of these results provide clear evidence of a leaky pipeline out of STEM fields when excluding 
biology, but strikingly not for biology itself. If anything, students appear to underestimate the rate 
they will ultimately study biology. For this reason and those previously mentioned, I focus on 
STEM fields excluding biology when studying my experimental results. 

Next, I leverage novel data I collect on students’ second choices of major to study whether there 
is evidence of a leaky pipeline into STEM fields. To study this, I perform similar analyses as before, 
comparing the distribution of second choice of major as measured in the survey to the distribution 
of majors three years after the intervention. For this second distribution, I restrict the sample to 
students who are not majoring in their top choice of major three years later. This comparison 
tells us whether students’ second choice of major matches up with the distribution of majors, for 
those who are not majoring in their top choice three years later. If there is no leaky pipeline into 
STEM, under certain assumptions, these distributions should be similar!!. Table 7 shows that 
while biology, social sciences and humanities all have higher representation in the distribution of 
students who switch out of their top field compared to the distribution of second choice of major, 
STEM sees a sharp decline. Using student’s subjective probabilities that they believe they will 
major in their second choice of major instead of the actual distribution of second choice of major 
yields a similar result. Students appear to over-estimate the rate they will switch into STEM, a 
result similar to those found in Stinebrickner and Stinebrickner (2013). Similar to the evidence of 
students switching out of STEM discussed earlier, I see this as evidence of a leaky pipeline into 
STEM. I now turn to whether receiving information about your ability in your top choice of major 


had any impact on tightening either of these leaky pipelines. 


'!'The primary assumption is that for any level of major switching, the individual transition probabilities from top 
choice of second choice of major do not favor any major over the other. Conditional on not majoring in your top 
choice of major and being in the control group, the distribution of majors is much more uniform than top choice; 
social sciences and STEM are very similar with 27.9% and 25.6% respectively, while Biology and Humanities are 
smaller at 20.9% and 11.6%. The low representation of biology in the switcher distribution is likely driven by the 
fact that biology is by far the most popular top choice of major and only 1.6% of students who select biology as their 
top choice of major select a biology field as their second choice. 


14 


4 Experimental Results 


To study whether the information provided to students in my intervention influenced STEM major 


choice, I estimate the following statistical model: 


STEM, =a+ PTREAT; + Xio¢t+ Siy+ mt & 


where STEM; is an indicator for whether the student is majoring in a STEM field three years 
after the intervention, TREAT; is an indicator if they receiving information in their top choice of 
major, X; is vector of background characteristics such as race/gender and standardized test scores, 
S; contains baseline survey responses and e«; is random error term. I also include an indicator 
for receiving information in your second choice of major, although I focus my interpretation on 


treatment in a student’s top choice of major’. 


Random assignment of students into treatment 
status assures me that E[e;|T RE AT;] = 0, allowing me to estimate the causal impacts of information 
on major choice decisions. 

To conduct inference, I estimate standard errors and calculate statistical significance under 
standard asymptotic assumptions. I also follow Athey and Imbens (2017) and perform randomiza- 
tion inference. In this procedure, treatment assignment is permuted across the analysis sample and 
and treatment effects are calculated for each permutation!?. Empirical p-values are then calculated 
by ranking these treatment effects and assessing where the true model lies within this distribution. 
This approach has many advantages for hypothesis testing. I present both standard errors and 
empirical p-values in my results. 

For each sample, I present two sets of results; one from a model that includes treatment indi- 
cators for both top and second choice of majors and; one which adds top major fixed effects and 


controls for demographic characteristics and survey responses. 


™ As the sample of students who were eligible for treatment in their top choice of major does not perfectly overlap 
with those eligible in their second choice, I create an indicator that equals one if you are in the latter but not the 
former. I then replace missing values for treatment in second choice of major with zeros and then include this indicator 
in the regression as well. Results are qualitatively very similar when excluding these variables in the analysis. 

18T perform 500 permutations for each outcome. 


15 


4.1 Main Effects on STEM Persistence 


Table 10 in the appendix presents results for students whose top choice of major was a STEM 
field'*. Focusing on these students captures the how well the treatment tightens the leaky pipeline 
of switching out of STEM fields. We see in columns one and two that treatment effects are large, 
representing an increase in persistence of about seven percent, although none of the estimates are 
significantly different from zero. This may be due to the fact that students who aspire to major 
in STEM fields are not considering non-STEM fields as alternatives. In this case, the treatment is 
unlikely to increase persistence, as this subset of students rarely choose non-STEM fields, even if 
they might switch majors within STEM. 

As we saw earlier, students place very little probability on graduating in a field that is neither 
their top nor second choice of major. I therefore use data on students’ second choice of major to 
characterize students who select a STEM field as their top choice of major, but also select a non- 
STEM field as their second choice of major as “marginal out STEM” students. These are the the 
students whose beliefs about their own ability in STEM fields are likely to play a larger role than 
those who are only considering STEM majors. Columns three and four of table 10 show results for 
these students. Here we find large and marginally significant effects of information on persistence, 
although when including the full set of controls the estimate becomes less precise. Putting this 
result into context, this tells us that learning you are above average in your top choice of (STEM) 


major increases persistence by 40 percent for this group. 


4.2 Main Effects on Switching into STEM 


As discussed earlier, I find evidence of a leaky pipeline into as well as out of STEM fields. To assess 
whether treatment increased switching into STEM fields, I estimate models studying students 
whose top choice of major is a non-STEM field. We see in columns five and six of table 10 that I 
cannot detect any significant patterns switching into STEM for this group. Similar to the effects on 


persistence discussed above, however, this may be explained by students who are not considering 


‘While the sample size for these models may seem small (n = 109), it is similar Stinebrickner and Stinebrickner 
(2013), who studied similar questions as I do here using a sample sizes of n = 139 and n = 93 when focusing on 
students considering science fields. 
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STEM at all not responding to treatment. These are students who select a STEM field for neither 
their top nor second choices of major. In this case, beliefs about ability may play little role in 
switching into STEM as STEM is not in their consideration set. To address this, I focus on 
“marginal into STEM” students, meaning those who select a non-STEM field as their top choice 
but select a STEM field as their second choice of major. Table 10 shows that learning you are 
above average in your top choice of major increases switching into STEM by 7 percentage points 
for this group. Given that the control mean for STEM major choice in this group is 1.6 percentage 
points, this represents an incredibly large treatment effect for this group. 

I next study whether receiving information that you are above average in your second choice of 
major induces switching when your second choice of major is a STEM field. Somewhat surprisingly, 
I fail to detect any significant effects when studying treatment in a student’s second choice of major 
when that choice of major is a STEM field. These results in columns five through eight in table 
10. While the effect of learning you are above average in your second choice of major for marginal 
in STEM students is large, these estimates are imprecise. What I can say in light of these results 
is that switching into STEM may only take place through learning one’s ability in one’s top choice 
of major, even when that major is not a STEM field. These results show that shocking beliefs 
in majors that are not a student’s most preferred major have a relatively weaker impact on the 
decision to switch into STEM. 

Lastly, table 10 also shows estimates from models that examine the effect of being treated in 
a student’s top choice of major on STEM major choice across the entire sample. These estimates 
provide the combined effect of both persistence within and switching into STEM fields. Here we 
see an increase in STEM major choice across the entire sample of 5 percentage points, representing 
an increase in STEM major choice of about a third compared to the control mean (although here 
only the model with the full set of controls is significant at the 10 percent level). Provided that 
learning you are above average in your top choice of major can influence STEM major choice in 
two distinct ways, these results provide a valuable reference point for how a policy designed to 
influence beliefs about ability may change STEM major choice across a sample of students with 


different preferences for majors. 
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4.3. Heterogeneous Treatment Effects 


I study potential heterogeneous treatment effects of my internvetion across three dimensions; gen- 
der, first generation college student status, and whether a student is eligible for the Pell Grant 


(low-income status). To do so, I estimate the following interaction model: 


STEM; =a+ §i\TREAT; + BoC; + BT REAT; * Ci + Xid+ Siy +Um + & 


where C; is one of the student characteristics mentioned above. Here, (G3 is the parameter of 
interests. These models estimate treatment effect heterogeneity for STEM major choice overall. 
Table 11 presents results for all three groups. 

Earlier I showed that students who selected a STEM field as their top choice of major were 
significantly less likely to be female. Somewhat surprisingly, we see the treatment did not differen- 
tially influence female students to major in STEM fields'’. Treatment effects for both low-income 
and first-generation students, however, are large and, in the case of first-generation students, pre- 
cisely estimated. Previous work as shown that first-generation college students typically have lower 
self-efficacy, a trait linked with persistence in difficult tasks Thompson (2021). As shown earlier, 
“leakers”, those who select a STEM field as their top choice of major but fail to persist in STEM, 
are seven percentage points more likely to be first-generation students (although this difference is 
not statistically significant). Using a large representative sample collected across several colleges, 
Thompson (2021) documents that high performing first-generation students are more likely to leave 
STEM compared to continuing-generation students. Here, the author shows this is likely due to 
first-generation students receiving negative shocks to their ability through lower grades in intro- 
ductory STEM courses. In light on this, my results show that learning about how they compare 
to their peers using alternative measures of ability is important for first-generation students when 
deciding on what to major to choose in college, particularly when considering STEM fields. I dis- 
cuss the relationship between the information I provide students and the information about ability 


contained in grades later in the paper when I compare my findings to the previous literature. 


Tn a sub-analysis, focusing on marginal into STEM students, I do observe large positive differential treatment 
effects on STEM persistence by gender. These results are not significantly different than zero, but are promising 
considering efforts to increase female persistence in STEM fields. Results available upon request from the author. 
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4.4 Mechanisms 


I next explore whether the information I provide students has a differential impact for students 
based on their initial beliefs about their relative ability. To do so, I estimate an interaction model, 
in which I add to the model used to study my main experimental results both a variable that 
incorporates students’ beliefs about the average score of recent graduates are and an interaction 
between treatment status and this variable. This variable is constructed by comparing the true 
value of the average score of recent graduates in a given major/measure pair with a student’s belief 
of that score. I create an indicator variable that captures if a student’s belief of the average was 


below the actual average: 


BD, = 1{ Actual AVEPOGE iia ce neasure) > Belief of AvEL AU Car imetalhayt 


Students who have BD; = 1 can be thought of as having underestimated the academic ability of 
their peers, relative to their own. Table 12 in the appendix presents results on STEM major choice 
for all students, marginal out and marginal into STEM students. For brevity, I present results from 
models that include a full set of controls. Here we see no differential effect on STEM major choice 
overall, nor for marginal out STEM students. I do find, however, a significant differential effect 
for marginal into STEM students. These students appear more likely to switch into STEM upon 
receiving treatment if they had originally underestimated the ability of their non-STEM peers. 

One way the information I provide may influence STEM major choice is by shifting students’ 
mean beliefs of peers’ ability towards the true value contained in the intervention. An alternative 
mechanism is through reducing the variance of students’ beliefs. Put another way, this would 
translate to reducing students’ uncertainty about their ability. Given that I find an increase in 
STEM persistence as a result of my intervention, the null results here for marginal out STEM 
students support the latter mechanism as ex-ante mean beliefs do not seem to influence persistence 
in light of new information about ability. 

I next study whether the information I provide students has a greater effect depending on how 


close a student’s own score is to the revealed true score. To do so, I construct a continuous “actual 
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difference” variable that captures how far a student’s own scores is from the revealed score: 


AD; = Own WCOTE taadon ieensure) — Actual UCT AOE (pacer piaaatie) 
— 


Actual Average(majormeasure) 


Unfortunately, I cannot discretize this variable as only students who had scores that were above 
the average were eligible for treatment. Therefore, all students would have a value of one for 
a variable that indicated if a student’s own score was above the revealed score. The continuous 
variable defined above, however, allows me to see how students responded to treatment differentially 
depending on how much higher their own score was from the revealed score. Table 12 shows us the 
estimates of this interaction term. In this case, a negative value would imply that the treatment 
effect is weaker for students with scores that are further away from the revealed true score. This 
also implies that students whose scores are closer to the true score experience a stronger treatment 
effect. This is in fact what I find. Coefficients for all three outcomes are negative. The differential 
effect for marginal out STEM students is economically large, but imprecise. Looking at marginal 
into STEM students, the differential impact of having a score much higher than the revealed score 


is again quite large and here is much more precisely estimated, as is the effect for the entire sample 


5 Conclusion and Discussion 


In this paper, I conduct an information intervention aimed at increasing the number of students 
majoring in STEM fields by changing their beliefs about their ability in their top fields of study. 
Using various measures, I find strong evidence of a leaky pipeline out of STEM that is consistent 
with a story where students leave STEM partly because they believe they are not academically 
suited for those fields. I also find evidence of a leaky pipeline into STEM, as students fail to 
switch into those majors at the same rates as other fields. In my experimental results, I find that 
providing STEM students with information that they are above average in that field increases 
STEM persistence among marginal out STEM students by 40% as well as increases switching 


into STEM among marginal into STEM students. Combining these two effects, the information 
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I provide increases STEM major choice by about 30% of the control mean. I also find that the 
effects are strongest for first-generation students. 

Few papers have directly assessed how beliefs about ability might factor into a student’s major 
choice decision. Stinebrickner and Stinebrickner (2013) use a novel panel data set containing 
information similar to what what I collect in the present study to assess how students update their 
perceptions of how likely they are to graduate with a degree in science. To do so, they model the 
belief updating process as a function of noisy signals students receive about their ability via grades 
they receive in science and non-science classes. They conclude that if students had entered college 
with the beliefs about graduating with a science degree they have later on, they would have been 
less likely to pursue science at the outset. Achieving this would effectively fix the leaky pipeline 
not by patching the leak but by turning down the water pressure. My results point to an slightly 
different interpretation. My findings are consistent with an updating process where students’ beliefs 
about their ability are a function not just of grades received in STEM classes, but of results from 
highly salient pre-collegiate academic performance measures, namely HS GPA and standardized 
test scores. Uncertainty about completing a STEM degree can be reduced by resolving uncertainty 
in either of these two domains. Even in my setting, it may be that low grades in STEM courses do 
take a toll on students perceptions of their ability, but those who received information that they 
are above average in that field have beliefs that may be more resilient to these negative shocks. 
Therefore, the effect of lower grades on beliefs about ability may be smaller. 

Previous work studying major choice and ability can also help to put the magnitudes of the 
effects I find here into context. I find that my treatment increased STEM major choice across 
the entire sample by about one third of the control mean. While this effect may seem large or 
implausible given the nature of the intervention, my results are in line with findings from other light- 
touch interventions on major choice outcomes. Studying how female students in an introductory 
economics course respond to information about their relative performance, Li (2018) finds that a 
combination of this information and a nudge to continue pursuing economics increases economics 
major choice by about 50 percent. Also studying persistence among female economics students, 


Porter and Serra (2019) find a 100 percent increase in economics majoring as a result of their light- 
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touch role model intervention. Despite the large increase in both STEM persistence and switching 
I find as a result of my treatment, they do not seem implausible in light of the results from these 
previous studies. 

The analyses I perform to study STEM major choice in this paper rely on unique measures of 
students’ consideration set of majors, as well as beliefs about how likely they are to major in those 
fields. Given that switching one’s major is a relatively frictionless choice at most U.S. universities 
and colleges, understanding both the extent and nature of the STEM leaky pipelines requires 
collecting these types of unique data, as students are not constrained to persist in their initial field 
of study. For example, as I find in my results, nudges aimed at increasing persistence in STEM fields 
are likely to be more effective if targeted towards students who are seriously considering majoring in 
non-STEM fields, (ie. marginal out or into STEM students). Being able to identify these students 
will help administrators, educators and policy-makers design more effective interventions. Lastly, 
as mentioned earlier in the paper, it would be extremely difficult to fully understand the nature 
of the leaky pipeline into STEM fields without the information collected as part of this project. 
Eliciting this broad set of student information helps provide a clearer picture of how students are 
thinking about their college major decisions as well as how to influence them. 

The supply and demand of STEM-capable workers in the U.S. has received much attention 
from academics and policy-makers alike over the past decade (Olson and Riordan, 2012; Carnevale 
et al., 2011). Empirical work studying the supply of STEM-workers has focused on educational 
pipelines from the classroom to the labor force and has shown that STEM students are more likely 
to switch out of STEM more so than students of other fields (Chen and Soldner, 2014). This 
comes at a time when the growth in demand for STEM workers appears to be growing faster 
than other occupations.!© The results from this paper highlight both potential reasons why the 
supply of STEM-capable workers is failing to keep up with demand and ways to address it. I find 
evidence that students’ beliefs about their ability to complete a STEM degree play a crucial role in 
STEM persistence in college. Moreover, these beliefs appear to be sensitive to nudges that provide 


information to students about their relative ability in their top fields of study. Information nudges 


‘Bureau of Labor Statistics: https: //www.bls.gov/emp/tables/stem-employment.htm 
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such as the one used in this paper may be powerful policy tools used in closing the gap between 
the supply and demand of STEM graduates. 

One limitation of my research design is that I do not elicit students’ preferences for different 
pieces of information directly. Using students’ actual academic measure selections, however, I 
can study whether students believed this information to be useful!’. I first study whether students 
were indifferent between selecting any of the five academic measures as being most representative of 
ability. There are two possibilities that I would view as consistent with this hypothesis; the majority 
of students selecting the first option that was displayed (HS GPA), or; students being equally likely 
to select any of the five measures!®. About half of students select HS GPA as most representative 
of ability in either their top or second choices of major (55% for top choice, 50% for second). While 
this is by far the most common selection, it does not comprise the overwhelming majority. Also, 
even among the remaining measures, I do not observe equal likelihoods of being selected. The 
remaining percentages range from SAT combined (19%) to SAT Reading and Writing (2.8%). Put 
together, I do not see this as evidence that students were indifferent about which measure they 
selected. 

An additional question is whether students simply selected the measure for which they were best 
at. To study this, I rank all students for whom I collect FERPA signatures across all five academic 
measures and then create percentage rankings for students for each measure. I then create a variable 
that captures in which academic measure students have the highest percentage rank. I find that 
only about a third of students select the measure for which they have the highest percentage rank 
as the measure that best represents ability in their top choice of major. This number is much closer 
to what we would expect if students were to select their top performing measure by chance (20%) 
than if they were systematically selecting measures in which they performed best. 

Lastly, if students did not consider the information I offered as part of my intervention useful, we 
would not expect students to select different measures for different majors. This would be true both 


across and within student selections. As mentioned earlier, students who selected a STEM field as 


'7T here use all survey respondents and do not restrict my analysis to students who were eligible for treatment in 
their top choice of major. This leaves me with 860 students when studying measures in students’ top choice of major 
and 847 when looking at their second choice. 

'8Unfortunately, I did not randomize the ordering of these measures in the survey. 
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their top choice of major were more likely to select the SAT math portion as most representative 
of ability in those fields. Students are even more likely to select SAT math when listing STEM 
as a second choice. I find similar patterns for the humanities and the SAT Reading and Writing 
portion. This measure is the second most chosen (after HS GPA) for humanities fields. When 
studying selections within students, I find that 35% of students select different measures when 
considering their top two choices of major. These results are generally consistent with student 
behavior in which these measures are carefully considered, rather than selected thoughtlessly. This 
careful consideration of measures may be what explains why students were likely to incorporate 
this information into their behavior. Finally, my findings also show that HS GPA is likely playing 
a large role in students perceptions of their ability, compared to the results of standardized tests. 

A further limitation of this project is that I do not collect measures of beliefs about ability 
after I provide information to students. I therefore do not observe the extent and manner in which 
students update their beliefs upon learning they are above average in their top choice of major. 
This is an important question as a central characteristic of virtually all educational institutions is 
the provision of performance feedback to students on a regular basis. As such, my results should 
be interpreted as reduced form effects of my treatment on STEM major choice. I leave to further 
research the task of providing a more complete picture of how students update their beliefs about 
ability when deciding on what major to choose. 

One additional, important limitation of my research design is that I cannot test how students 
who are below average would respond to information. This is an important limitation as prior 
research has demonstrated that these students may respond negatively to such information (Franco, 
2020; Azmat et al., 2019). Understanding how these students might respond to learning they are 
below average would help policy-makers balance the trade offs when considering making average 
scores of recent graduates in different majors public. Below average students may increase their 
study effort in response to learning they are in fact below average. Other work in the performance 
feedback literature shows that this may be the case (Azmat et al., 2019). Also, there may be savings 
from informing students they are below average and inducing them to change majors, as courses 


that do not contribute to a degree or skills can be costly for both the student and the institution. 
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Lastly, there continues to be a dramatic under representation of female students in STEM fields. 
Disappointingly, I do not find that my treatment had a differential impact on STEM major choice 
for female students. In contrast, my results for first-generation students demonstrate that this 
subgroup is likely poised to use information about performance of other college students that is 
not publicly available. These findings highlight a possible way to address a particurly accute leaky 
pipeline out of STEM for this group (Thompson, 2021). 

Taken together, my results demonstrate the importance of learning about one’s own ability 
relative to other students when decided whether to continue or switch into STEM fields. Information 
about the academic measures I discuss here are virtually cost less to institutions of higher education. 
For those concerned with the increasing the rate of STEM major choice and completion at U.S. 
colleges, these findings may provide a road map for how they may be used to support students in 


that endeavor. 
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Table 1: Selection Regressions 


(1) (2) 


Treat (Top Choice) Treat (2nd Choice) 
b/se b/se 
Biology Top Choice 0.054 0.080 
(0.109) (0.119) 
STEM Top Choice 0.060 -0.098 
(0.114) (0.124) 
Soc Sci Top Choice 0.067 0.088 
(0.113) (0.123) 
Pr(Graduate Top Choice) 0.094 -0.135 
(0.094) (0.098) 
E[Salary Top Choice] 5y 0.000 0.001 
(0.001) (0.001) 
Believe Good (Top Choice) 0.012 -0.089 
(0.054) (0.057) 
Female 0.066 -0.085 
(0.051) (0.055) 
Black -0.243 -0.330 
(0.200) (0.220) 
White -0.161 -0.158 
(0.121) (0.130) 
Asian/Asian American -0.085 -0.134 
(0.117) (0.126) 
Hispanic -0.066 -0.104 
(0.129) (0.142) 
Low-Income 0.051 -0.009 
(0.067) (0.070) 
First Generation -0.074 0.002 
(0.059) (0.062) 
SAT/ACT score 0.000 0.000 
(0.000) (0.000) 
Freshman -0.054 0.011 
(0.050) (0.053) 
Missing Demographic Vars 0.070 -0.012 
(0.082) (0.094) 
Observations 485 429 


Notes: Each column represents a regression of all row variables onto treatment assignment. For column one, this 
sample consists of the analytic sample studied throughout the paper. As discussed in the experimental section, the 
number of observations in column two does not match up perfectly with the number of observations in column one. 
This is because the overalp between treatment in top choice and second choices of major was not perfect, as some 
students were above average, and therefore eligible for treatment, in their top choice but not their second choice of 
majo. I zero out missing treatment status values for second choice of major and add an indicator that equals one if a 
student is missing a value for that variable when studying treatment effects in top choice of major. Standard errors 
are in parentheses. 
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Table 2: Sample Descriptive Statistics 


(1) 2) 3) @) 
All students Top non- STEM ‘Top STEM Difference 
Earnings 5y post grad (thousands) $78.52 $76.04 $86.98 $10.94 
(3.27) 
Earnings 20y post grad (thousands) $113.13 $110.70 $121.42 $10.72 
(3.31) 
Prob grad Top Major 72.01 72.02 71.98 -0.04 
(0.01) 
Believe good Top Major 69.15 69.51 67.89 -1.62 
(0.32) 
HS GPA top measure 68.53 69.52 65.13 -4.5 
(0.87) 
SAT math top measure 8.70 6.68 15.60 8.91 
(2.94) 
Female 66.87 73.80 43.12 -30.68 
(6.21) 
First-Gen 31.68 32.89 27.52 -5.36 
(1.06) 
Low-Income 21.11 20.32 23.85 3.53 
(0.79) 
Black 2.07 2.41 0.92 -1.49 
(0.96) 
Asian 48.65 47.06 54.13 7.07 
(1.30) 
Hispanic 15.73 15.78 15.60 0.18 
(0.45) 
White 29.19 30.49 24.77 -5.71 
(1.15) 
Observations 483 374 109 


Notes: The sample here consists of students who were eligible for treatment in their top choice of major and signed 
the FERPA release (so that I can observe their major choice decisions). Numbers represent either means for each 
variable. Column 2 restricts students who selected a non-STEM field as their top choice of major, while column 3 
does the same for students who selected a STEM field. Column 4 reports both the difference between columns 2 and 
3 as well as tests for statistical differences between the samples. t-statistics are in parentheses. 


31 


Table 3: Majors and Academic Measures 


(1) (2) 


Top Choice Second Choice 
Majors 
Biology 47.62% 11.33% 
STEM (No bio) 22.57% 41.13% 
Social Sciences 24.64% 30.30% 
Humanities 5.18% 17.24% 
Academic Measures 
HS GPA 68.53% 54.09% 
SAT Combined 10.56% 16.36% 
SAT Math 8.70% 15.30% 
SAT Reading/Writing 0.62% 6.86% 
ACT 11.59% 7.39% 
Observations 483 406 


Notes: The sample here consists of students who were eligible for treatment in their top choice of major, signed the 
FERPA release (so that I can observe their major choice decisions) and who have valid selections for second choice of 
major. The bottom right panel (academic measure for second choice of major) only has 379 observations. The lack 
of perfect overlap in number of observations comes from the fact that not all students answered the second choice of 
major questions. 


Table 4: Leaky Pipeline out of STEM #1 


(1) (2) (3) (4) 


Top choice Prob. Empirical Prob. A Percentage A 

Biology 46.86% 48.95% +2.09 +4.46% 
(50.0) (49.8) 

STEM (No bio) 22.59% 15.06% -7.53 -33.33% 
(41.9) (37.7) 

Social Sciences 24.69% 28.03% +3.34 +13.52% 
(43.21) (45.01) 

Humanities 5.86% 5.86% +0.0 + 0% 
(23.53) (23.53) 

Observations 239 239 


Notes: The sample here consists of students who were assigned to control in their top choice of major and signed 
the FERPA release (so that I can observe their major choice decisions). Column one provides the distribution of 
top choice of major by the four major groupings. Column two presents the distribution of actual major selections 
three years later. Column three takes the difference between columns one and two, while column four divides this 
difference by the value found in column one. 
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Table 5: Leaky Pipeline 


out of STEM #2 


(1) 


(2) 


Pr(Persist 2ys Later) PP Drop 
Biology (n = 112) 91.96% -8.04% 
(27.03) 
STEM (No bio) (n = 54) 61.11% -38.89% 
(49.21) 
Social Sciences (n = 59) 88.14% -11.86% 
(32.61) 
Humanities (n = 14) 57.14% -42.86% 
(51.36) 


Notes: The sample here consists of students who were assigned to control in their top choice of major and signed the 
FERPA release (so that I can observe their major choice decisions). Each row restricts the sample to those students 
who selected a major in the indicator major group as their top choice of major. Column one presents the average 
probability students placed on graduating in their top choice of major. Column one shows the percentage of students 
who are majoring in their top choice of major three years later for reach of these major groups. Column two presents 


this number minus 100. 


Table 6: Leaky Pipeline out of STEM #3 


(1) 


(2) 


(3) 


(4) 


Subjective Prob. Empirical Prob. A Percentage A 

Biology (n = 112) 73.27% 91.96% +18.69 +25.50% 
(23.57) (27.31) 

STEM (No bio) (n = 54) 70.48% 51.85% -18.63 -26.43% 
(25.39) (50.43) 

Social Sciences (n = 59) 68.31% 83.05% +14.74 +21.57% 
(26.09) (37.84) 

Humanities (n = 14) 57.71% 50.0% -7.71 -13.35% 
(20.81) (51.89) 


Notes: The sample here consists of students who were assigned to control in their top choice of major and signed the 
FERPA release (so that I can observe their major choice decisions). Each row restricts the sample to those students 
who selected a major in the indicator major group as their top choice of major. Column one presents the average 
probability students placed on graduating in their top choice of major. Column two shows the actual distribution of 
major choices three years later. Column three takes the difference between columns one and two, while column four 
divides that difference by the value found in column one. 
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Table 7: Leaky Pipeline into STEM #1 


(1) (2) (3) 
Sec choice Prob. Empirical Prob. A 


Biology 20.93 32.56 11.63 
(41.16) (47.41) 

STEM (No bio) 25.58 6.98 -18.60 
(44.15) (25.78) 

Social Sciences 27.91 34.88 6.97 
(45.39) (48.22) 

Humanities 11.63 13.95 2.32 
(32.44) (35.06) 

Observations 43 43 


Notes: The sample here consists of students who were assigned to control in their top choice of major, signed the 
FERPA release (so that I can observe their major choice decisions) and who have valid selections for second choice 
of major. I also restrict the sample here to students who did not major in their top choice of major. Column one 
shows the distribution of second choice of major for this group, while column three shows the distribution of actual 
major choices three years later. Column three takes the difference between columns one and two. 


Table 8: Leaky Pipeline into STEM #2 


(1) (2) (3) (4) 


Subjective Prob. Empirical Prob. A % A 

2nd Biology (n = 20) 17.40% 40.00% +22.60 +129.88% 
(13.61) (50.26) 

2nd STEM (No bio) (n = 87) 19.87% 6.90% -12.97 65.27% 
(16.77) (25.49) 

2nd Social Sciences (n = 70) 22.66% 20.00% +2.66 +11.73% 
(20.27) (40.29) 

2nd Humanities (n = 30) 12.63% 3.33% -9.33 -73.63% 
(14.23) (18.26) 


Notes: The sample here consists of students who were assigned to control in their top choice of major, signed the 
FERPA release (so that I can observe their major choice decisions) and who have valid selections for second choice 
of major. Each row consists of the set of students whose second choice of major is in the indicated major group. The 
first row represents the average probability those students placed on majoring in their second choice of major. The 
second row shows the percentage of students who actually selected a major in their second major group. Column 
three takes the differences between columns one and two, while column four divides that differences by the value in 
column one. 
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Table 9: Comparison of STEM Leakers and Persisters 


(1) (2) (3) 


Persister Leaker Difference 


SAT Math score 691 705 14 
(0.50) 
Female 27.27 57.14 29.87 
(2.26) 
First-Generation 21.21 38.10 16.88 
(1.35) 
Low-Income 15.15 23.81 8.66 
(0.78) 
Asian 54.55 38.10 -16.44 
(1.17) 
Hispanic 15.15 28.57 13.42 
(1.19) 
White 27.27 23.81 -3.46 
(0.24) 
Believe Good 63.63 61.90 1.73 
(0.13) 
(Own Score - Belief) /Belief CAT 3.78 -3.34 
(1.13) 
(Actual - Belief) /Actual -0.44 -3.37 -2.94 
(1.21) 
Observations 33 21 


Notes: The sample here consists of students who were assigned to the control group for treatment in their top choice 
of major, who selected a STEM field as their top choice of major but ultimately chose to switch out of STEM.t 
statistics in parentheses. 
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Table 10: The effect of treatment on STEM Major Choice 


(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) 
Top=STEM Top=STEM Marginal out Marginal out Sec=STEM Sec=STEM Marginal In MarginalIn Entire Sample Entire Sample 
Treat (Top Choice) 0.069 0.046 0.235 0.203 0.068 0.027 0.070 0.075 0.057 0.055 
(0.092) (0.093) (0.125) (0.145) (0.068) (0.053) (0.042) (0.044) (0.035) (0.026) 
[0.478] [0.638] [0.080] [0.200] [0.336] [0.590] [0.080] [0.084] [0.116] [0.062] 
Treat (Second Choice) -0.027 0.039 0.038 0.025 
(0.078) (0.059) (0.048) (0.049) 
[0.702] [0.520] [0.378] [0.584] 
Control Mean 0.611 0.611 0.517 0.517 0.288 0.288 0.016 0.016 0.151 0.151 
Observations 109 109 58 58 167 166 116 115 483 481 
FE+Controls N Y N Y N Y N Y N Y 


NoteThis table presents results from equation (1) in the text. The outcome of interest is whether a student is majoring in a STEM field three years after the intervention. Models without fixed 
effects and controls include indicators for treatment in top and second choice of major, as well as an indicator for whether a student is missing a valid value for treatment in their second 
choice of major. Odd numbered columns include indicators for whether a students top choice of major was a Biology, STEM or Social Science field, as well as the full suite of demographic and 
survey response controls. These include indicators for gender, race/ethnicity, SAT/ACT scores, first generation and low income status. Survey responses include expected pecuniary returns in 
one's top choice of major, the probability they believe they will graduate in their top choice of major and whether they believe they are above average in their top choice of major: 


Table 11: Heterogeneous effects by gender, low-income and first generation status on STEM Major 
Choice 


(1) (2) 


Recent = STEM Recent = STEM 
Female x Treat -0.018 -0.032 
(0.070) (0.056) 
0.826 0.636 
Low Income x Treat 0.130 0.078 
(0.085) (0.065) 
0.120 0.256 
First Gen x Treat 0.130 0.111 
(0.074) (0.056) 
0.056 0.066 
Observations 483 483 
FE+Controls N Y 


Notes: Both columns study the differential effect of treatment on the student characteristic of interest on STEM major 
choice for the entire analytical sample. Each row shows the coefficient of the interaction term including treatment in 
top choice of major with the characteristic of study. Models also include indicators for treatment status in top choice 
of major as well as an indicator for the student characteristic. Standard errors are in parentheses, empirical p-values 
are in brackets. 
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Table 12: Studying Mechanisms and STEM Major Choice 


) 2) (3) 


Marginal Out Marginal In Entire Sample 
BD x Treat 0.040 0.021 0.204 
(.053) (.329) (.084) 
[.482] [.948] [.030] 
AD x Treat -0.421 -1.023 -0.763 
(2.348) (0.692) (.436) 
[.862] [0.12] [.012] 
Observations 483 58 116 
FE+Controls Y Y Y 


Notes: All columns study the differential effect of treatment based on students’ beliefs and their own average score. 
Columns represent models looking at whether a student’s most recent major is a STEM field based on three different 
samples; marginal out students; marginal in students; and the entire analytical sample. Each row shows the coefficient 
of the interaction term including treatment in top choice of major with the characteristic of study. Models also include 
indicators for treatment status in top choice of major as well as an indicator for the student characteristic. Standard 
errors are in parentheses, empirical p-values are in brackets. 
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Figure 1: Probability of Graduating in Top or Second choice or some other major 


8 
Probability of majoring in top major = 71.9% Ha 
6+ 
4 
Probability of majoring in second major = 18.3% 
Probability of majoring in other major = 9.8% [1 
0 7 


Note: The sample here consists of students in the analytic sample. This implies that the sample 
size for this analysis is n = 438. 


Note: The sample here consists of all students answering the following survey question: "What is the probability you 
believe you will graduate in your top choice of major, your second choice of major, or some other major (remember 
that the sum of the probabilities must add up to 1)" 
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Figure 2: Beliefs about Ability (STEM Top Choice) 


0 es A 
(Your ability) - (Beliefs about average ability of graduates) 
Sane Actually good t———— All students 


Note: The sample here consists of all students who selected a STEM field as their top choice of major. The 
distributions are of the difference between one's own score of their selected measure for their top choice of 
major and the actual average score of recent graduates in that major. The second plot does the same but for 
the students whose own score is actually above the average score. 
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